This invention is in the field of telecommunications, and is more specifically directed to the decoding of video signals communicated over relatively low bit-rate communications lines.
As is evident in the art, the extent and rate of telecommunication of information around the world, particularly by way of the Internet, has greatly increased in recent years. In particular, the high bit rates at which modern communications systems operate have permitted the communication of a wide range of information, from simple text, to audio signals, to graphics images, and up to and including motion picture video information. Data compression techniques, in combination with the improvement in available bit rate, have also enabled such communication, particularly of video information. As a result, end users have become accustomed to receiving information of all of these types, at data rates and display quality that approaches "real-time" viewing of the information, particularly at workstations that are connected to high-speed digital communications networks. This ability to display such information, particularly motion picture video information, in real time has resulted, of course, in a substantial increase in the production and distribution of such video information, as motion picture video is often a preferred medium for communication of information.
However, high-speed telecommunication facilities are not available in all locations, or in all applications. For example, most home computer systems effect telecommunication over analog telephone lines, using modems of bit rates up to on the order of 28.8 and 56.6 kbps. The use of mobile computing devices, such as notebook computers, to effect telecommunications is also increasing at a dramatic rate, with the telecommunication of information being carried over cellular and wireless links. In response to this demand, video compression approaches have been developed to serve the need of communication of motion picture information over low bit-rate (less than 64 kbps) facilities. Well-known examples of these video compression techniques include the H.261 and H.263 video compression standards, developed in conjunction with the International Telecommunications Union (ITU) and based upon the well-known MPEG standards developed by the Moving Pictures Experts Group. These approaches have enjoyed popularity in the communication of compressed video information over these low bit-rate facilities. Additionally, compression according to an H.263-like algorithm is included within the currently-proposed MPEG4 standard.
The receipt of compressed video information, including information compressed according to the H.263 video compression standard, requires the receiving workstation or computer to perform video decoding of the communicated information, in order to effect display of the motion picture. Such video decoding operations conventionally include the execution of data processing instructions, including memory read and write accesses, by the receiving computer.
Referring now to FIGS. 1 and 2, the operation of a conventional video decoding system in receiving and decoding compressed video information, according to the H.263 standard, will now be described by way of further background. As is fundamental in the art, H.263 video decoding predicts motion from frame to frame in a video sequence by way of motion vectors, which are two-dimensional vectors that provide offsets from the coordinate position from a prior, reference, frame to coordinates in the frame currently being decoded. The redundant, or non-moving, information from frame-to-frame is encoded by way of a transform, for example a discrete cosine transform (DCT), the inverse of which is performed by the video decoding system. The inverse transform of the redundant portion of the transmitted frame, in combination with the results of the motion vectors, produce a displayable frame.
FIG. 1 illustrates the general nomenclature and temporal sequence utilized in the generation of predicted frames in H.263 video decoding. According to the H.263 standard, the incoming bitstream of video data corresponds to displayable video frames which are predicted over time, using the currently decoded frame and the previously decoded frame in the sequence. As illustrated in FIG. 1, frame P.sub.T-1 is a previously predicted frame corresponding to the immediately preceding frame in the video sequence to that currently being decoded, and is referred to as a "P" frame, as it is predicted from its prior neighbor frame. Past P-frame P.sub.T-1 is then used to predict a future (or current) P frame P.sub.T ; as will be described hereinbelow and as known by those in the art familiar with the H.263 standard, the forward-prediction of frame P.sub.T is based upon past P frame P.sub.T-1 and upon motion vectors derived from the frame currently being decoded. According to the H.263 standard, a forward prediction from the past P-frame P.sub.T-1 and a backward prediction from the future P-frame P.sub.T are used to predict a current B frame B.sub.T (the "B" indicating that frame B.sub.T is bidirectionally predicted). The combination of the decoded P frame P.sub.T and B frame B.sub.T is referred to as the "PB" frame in the H.263 standard, and serves as the basis of the displayed frame. The H.263 standard is thus particularly well-suited for low bit rate telecommunication channels, as its coding scheme is intended to take advantage of temporal and spatial redundancy.
A conventional algorithm for performing the generation of the decoded P frame P.sub.T and B frame B.sub.T is illustrated by way of the pictorial representation of FIG. 2. This implementation corresponds to a C language computer program (Version 1.3) available from Telenor at http://www.nta.no/brukere/DVC/h263.sub.-- software/ (February, 1995), and includes the extensions of allowing motion vectors to point outside of the frame (i.e., unrestricted motion vectors) and of using extra motion vectors to compensate for motion according to the overlapped block motion compensation (OBMC) approach. In this exemplary implementation, input bitstream IBS includes a stream of data which is initially decoded in process 2 to generate motion vectors MV, and uncompensated decoded macroblocks 3, 3'. Decoded macroblock 3 corresponds to a P block of information from input bitstream IBS, while decoded macroblock 3' corresponds to a B block of information from input bitstream IBS. Elements MV, 3, 3' of the input bitstream are to be used, in combination with previous P-frame P.sub.T-1, in the prediction of the decoded frames P.sub.T and B frame B.sub.T. As is known in the art, each frame (P-frame or B-frame) includes multiple macroblocks, typically arranged in the QCIF format as nine rows of eleven macroblocks; each macroblock in the H.263 standard corresponds to six blocks of DCT coefficients derived from an eight-by-eight array of pixels; these six blocks generally include four blocks of luminance (Y) data and two blocks of chrominance (Cr, Cb) data.
FIG. 2 illustrates that the previous P-frame P.sub.T-1 is stored in memory as array signal oldframe. In process 8, the video decoding system adds a border of image blocks (e.g., forty-four macroblocks) around the perimeter of previous P-frame P.sub.T-1 ; these border image blocks permit so-called "unrestricted" motion vectors, in that the motion vectors may extend to outside of the viewed image area, resulting in a decompressed image that is less susceptible to visible edge effects. The resulting bordered P-frame Pext.sub.T-1 is stored in memory as array signal edgeframe, as illustrated in FIG. 2.
As illustrated in FIG. 2, each macroblock of bordered P-frame Pext.sub.T-1 is applied to forward P prediction process 9 which, in combination with motion vectors MV from decoding process 2 generates intermediate P frame Pnew.sub.T, which is stored in memory as array signal newframe. FIG. 3 illustrates, by way of a flow diagram, the operation of conventional forward P prediction process 9, in the case where OBMC motion compensation is enabled. As illustrated in FIG. 3, forward prediction process 9 begins with the retrieval, in process 9a, of bordered prior frame Pext.sub.T-1, including its luminance (Y) and chrominance (Cr, Cb) information. An OBMC index i is initialized to zero in process 9b, and four passes of motion compensation upon the luminance data in the prior frame Pext.sub.T-1 is then performed. In conventional video decoder systems, the OBMC motion compensation is performed by calling and applying a function to the luminance data of prior predicted frame Pext.sub.T-1, using motion vectors MV, for the index i, as indicated by process 9c in FIG. 3. Processes 9d, 9e increment and test the index i as shown, to determine the completion of the luminance OBMC compensation routine. For OBMC compensation of luminance data, processes 9b through 9e are generally performed on a macroblock basis, with each of the four luminance blocks processed one at a time through the loop of processes 9c and 9d. Following decision 9e determining that the loop of processes 9c, 9d is complete (decision 9e being YES), process 9f then performs motion compensation upon the upon the chrominance Cr information in the bordered prior frame Pext.sub.T-1, using motion vectors MV, and process 9g similarly performs motion compensation upon the chrominance Cb information. The resulting predicted frame Pnew.sub.T is then stored as array signal newframe.
As is known in the art, motion compensation modes other than OBMC are utilized in H.263 video compression and decompression. These alternative modes are similar to, but simpler than, the OBMC mode described above; conventional algorithms for these simpler motion compensation modes are known in the art.
In conjunction with forward P prediction process 9, each decoded P macroblock 3 is applied to inverse discrete cosine transform (IDCT) process 4, producing a transform P block 5 corresponding to the redundant spatial domain frame-to-frame information; this transform P block 5 is summed with corresponding macroblocks of predicted intermediate frame Pnew.sub.T in process 10 to produce future predicted frame P.sub.T. Frame P.sub.T replaces predicted intermediate P frame Pnew.sub.T in memory as array signal newframe, as shown in FIG. 2.
Bordered past predicted frame Pext.sub.T-1 is also applied to forward B prediction process 12 to generate first intermediate B frame B1.sub.T, which is stored in memory as array signal Bframe. FIG. 3 also illustrates the details of forward B prediction process 12 which is performed after forward P prediction process 9 described hereinabove. As is known in the art, the four-vector OBMC compensation process is not applied in the generation of B frames. Process 12 begins with the retrieval, in process 12a, of the prior predicted and bordered P frame Pext.sub.T-1, including the luminance and chrominance information. Processes 12b, 12c, 12d apply motion compensation to the luminance, chrominance Cr, and chrominance Cb information from bordered P frame Pext.sub.T-1, using the motion vectors MV from decoding process 2, typically in a macroblock-by-macroblock manner. The resultant B frame B1.sub.T is then stored in memory as array signal Bframe, in process 12e.
Backward B prediction process 14 retrieves frame B1.sub.T from array signal Bframe in memory and future predicted frame P.sub.T from array signal newframe in memory, and combines these retrieved frames B1.sub.T, P.sub.T to produce second intermediate B frame B2.sub.T, which replaces first intermediate B frame B1.sub.T in memory as array signal Bframe. Decoded B macroblocks 3' are sequentially applied to IDCT process 4' to produce transform B macroblocks 5' which is added to corresponding macroblocks of second intermediate B frame B2.sub.T in summing process 16, the result of which is current B frame B.sub.T. Frame B.sub.T is stored in memory as array signal Bframe, replacing second intermediate B frame B2.sub.T, as shown in FIG. 2.
As noted above, the PB frame corresponding to the combination of predicted frames P.sub.T and B.sub.T are used to derive the displayed frame, decoded from input bitstream IBS. Decoding of the next received frame is then performed in similar manner, with the current P frame P.sub.T becoming previous P frame P.sub.T-1 for the next decoding operation; this swap is conventionally performed simply by swapping the pointers for array signals newframe and oldframe, as frame P.sub.T-1 from the generation of P frame P.sub.T is no longer needed.
It has been observed, however, that the large quantity and high rate of information received in connection with compressed video, and the processing requirements for decoding the information, can result in substantial power dissipation by the receiving computer. Power dissipation is especially of concern in mobile computers, such as notebook computers, which are powered by batteries. In particular, the heavy rate at which complex data types are accessed from main memory resources (typically off-chip memory), in decoding and displaying compressed video information, has been observed to be a significant drain of battery power, thus reducing the usability of such mobile computers in receiving and displaying decoded video information.
In particular, it has been observed that conventional video decoding algorithms have been written with ease of program control being of paramount importance. However, as is evident from the above description of a conventional H.263 process, repeated memory accesses are performed to effect the motion compensation and prediction operations. For example, referring to FIGS. 2 and 3, each pixel in array signal edgeframe is written once (by process 8) and read twice (in processes 9, 12) for each decoded frame. Similarly, referring to FIG. 2, each pixel in array signal Bframe is written three times (in processes 12, 14, 16) and read twice (in processes 14 and 16) for each decoded frame, and each pixel in array signal newframe is written twice (in processes 9, 10) and read once (in process 14).
In this regard and by way of further background, the impact of memory-related power consumption in video and image processing is described in F. Catthoor, et al., "Global Communication and Memory Optimizing Transformations for Low Power Signal Processing Systems", VLSI Signal Processing VII, (IEEE, 1994), pp. 178-187. This paper further discloses that transformations of the video decoding specifications can obtain significant power reduction in and of themselves. Examples of transformations to optimize memory access operations, from the standpoint of power reduction, are disclosed. However, the implementation of these concepts into a particular video decoding scheme, such as those based on the H.263 standard, is not discussed in this article; it is believed that such implementation has not heretofore been disclosed or effected in the art.